S03-359 PATENT APPLICATION 

What is claimed is: 

1. A method for capturing, synchronizing, and replaying a sketching activity and media 
information associated to said sketching activity, said method comprising: 

a) simultaneously timestamped capturing said sketching activity and said media 
information; wherein 

said sketching activity producing one or more sketch objects, each 
having a corresponding sketch object timestamp, wherein 

said media information includes audio data, and wherein 
said audio data are captured in an audio file; 

b) transcribing said audio file, recording respectively recognized keywords or 
phrases and their corresponding timestamps; 

c) converting all timestamps associated to said one or more sketch objects, said 
audio file, and said keywords or phrases to a common time base; 

d) enabling a user to select a starting point for replay, said starting point is one or 
more of said sketch objects, a keyword, or a phrase; 

e) based on said starting point, synchronizing said one or more sketch objects, 
said audio file, and said keywords or phrases, utilizing their respective 
corresponding common base timestamps; and 

f) replaying said sketching activity and said media information based on said 
starting point. 

2. The method according to claim 1, wherein 

sketch object timestamp = raw sketch object timestamp - session start timestamp; 
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audio timestamp = raw audio timestamp * 1000; 

keyword timestamp = system clock keyword timestamp - session start timestamp; 

transcribed data timestamp = (Tr*Ds/Dr) + Tsst, where 

Tr = raw transcribed data timestamp - raw start time, 

Ds = system clock session end time - system clock session start time, 

Dr = raw end time - raw start time, and 

Tsst = system clock applet start time. 

The method according to claim 2, further comprising: 

comparing said sketch object timestamp with a keyword timestamp associated 
with said starting point; and 

replaying said sketch activity and said media information starting from the 
latest sketch object drawn before said keyword was spoken. 

The method according to claim 3, further comprising: 

synchronously displaying text corresponding to said audio data, 
synchronously playing video data corresponding to said sketching activity, 
synchronously playing video data corresponding to said audio data, or a combination 
thereof. 

The method according to claim 1, further comprising: 

importing a background image of which said sketch activity annotates. 
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The method according to claim 1, further comprising: 

automatically indexing and storing said sketch activity and said media 
information in a database. 

The method according to claim 6, further comprising: 

distributing via real time Internet streaming said sketch activity and said 
media information over a computer network; wherein said timestamped capturing 
occurs at a first computer connected thereto and said replaying occurs at a second 
computer connected thereto. 

The method according to claim 7, further comprising: 

enabling a user of said second computer access to said sketch activity and said 
media information via an interactive graphical user interface. 

The method according to claim 8, wherein 

said interactive graphical user interface and said database are maintained by a 
server connected to said computer network. 

The method according to claim 7, wherein 

said computer network is characterized as an intranet, the Internet, or a 
combination thereof, said computer network comprising wired and wireless 
communication links. 
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11. A digital computer system programmed to perform the method of claim 1. 

12. A computer-readable medium storing a computer program implementing the method 
of claim 1. 



13. A system for real time capturing and synchronous replaying a sketching activity and 
related media information, said system comprising: 

means for capturing said sketching activity and said media information 
simultaneously; wherein said media information includes audio signals and video 
15 signals and said sketching activity producing a plurality of sketch entities; 

means for encoding said sketch entities with corresponding sketch object 
timestamps and said media information with corresponding media timestamps; 

means for converting said sketch object timestamps to sketch object common 
base timestamps and said media timestamps to media common base timestamps; 
20 means for comparing said sketch object common base timestamps and said 

media common base timestamps; and 

an interactive graphical user interface for synchronously replaying said sketch 
activity and said media information; 

25 14. The system of claim 13, further comprises: 

means for transcribing said audio signals into recognizable words and phrases; 
means for associating said recognizable words and phrases with 
corresponding keyword timestamps; and 
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means for converting said keyword timestamps to keyword common base 
timestamps. 

The system of claim 13, further comprises: 

means for enabling a user to select a keyword from said recognizable words 
and phrases; and 

means for comparing said sketch object common base timestamps with a 
keyword common base timestamp associated with said selected keyword and 
determining a starting point from when said interactive graphical user interface 
replays simultaneously and in synchronization said sketch activity, said media 
information, and a text representation of said media information. 

The system of claim 13, further comprising: 

a database for storing said sketch activity and said media information; 

means for distributing said sketch activity and said media information over a 
computer network; 

means for enabling one or more user units distributed in said computer 
network to modify said sketch activity and said media information; and 

means for updating said database in accordance with said modification. 

The system of claim 16, further comprising: 

a server for maintaining said database and said interactive graphical user interface. 
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5 18. A computer-readable medium storing a computer program implementing the system 
of claim 13, wherein said interactive graphical user interface comprises a color pallet, 
a virtual canvas, and a plurality of functionalities. 

19. The computer-readable medium of claim 18, wherein said functionalities comprise: 
10 means for removing the last drawing movement from said canvas; 

means for removing all drawing movements on said canvas; 

means for retrieving and importing a background image onto said canvas; 

means for removing said background image; 

means for retrieving, importing and displaying slides; 
15 means for placing a 2 dimensional or a 3 dimensional grid on said canvas; 

means for storing said sketch activity as a subpage; 

means for importing said subpage onto said canvas; 

means for importing drawing movements of said subpage onto said canvas; 

means for tracing said drawing movements in a buffer; 
20 means for placing content of said buffer onto said canvas; 

means for taking a screenshot of said canvas; and 

means for placing said screenshot on said canvas as background. 

20. The computer-readable medium of claim 19, wherein each of said functionalities is 
25 activated and deactivated via a voice command or a screen representation 

characterized as a button. 



